FLUX Second Wave! FLUX-GGUF Quantification, the free abundant FLUX-API is here!

Introduction

Although quantization is not suitable for conventional UNET models (conv2d), transformer/DiT models like flux seem to be less affected by quantization. This allows them to run in a quantized manner on low-end GPUs. Recent GGUF quantization supports native ComfyUI and FLUX models.

Model link:

https://modelscope.cn/models/AI-ModelScope/FLUX.1-dev-gguf

ComfyUI code link:

https://github.com/city96/ComfyUI-GGUF

Due to FLUX’s high requirements for graphics cards, DashScope has also launched the FLUX-API, which supports Chinese input, is free and generous, making it very convenient for users who are new to FLUX.

API documentation link:

https://help.aliyun.com/zh/dashscope/developer-reference/flux-apis

ComfyUI code link:

https://github.com/modelscope/comfyscope/

Magic Integration Creative Space:

https://modelscope.cn/studios/AI-ModelScope/flux

Best Practices for FLUX Based on ComfyUI

In addition to directly experiencing it on the Magic Dock Creative Space, we offer hands-on experience with the FLUX-GGUF model based on ComfyUI and the FLUX-API. ComfyUI is a powerful and highly modular user interface and backend for AIGC graphics and video generation. This article uses ComfyUI to experience the use of the FLUX model on the free GPU Notebook provided by the Magic Dock community:

Download and deploy ComfyUI

Clone the code and install the relevant dependencies:

#@title Environment Setup

from pathlib import Path

OPTIONS = {}
UPDATE_COMFY_UI = True  #@param {type:"boolean"}
INSTALL_COMFYUI_MANAGER = True  #@param {type:"boolean"}
INSTALL_CUSTOM_NODES_DEPENDENCIES = True  #@param {type:"boolean"}
INSTALL_FLUX = True
INSTALL_FLUX_API = True  #@param {type:"boolean"}
OPTIONS['UPDATE_COMFY_UI'] = UPDATE_COMFY_UI
OPTIONS['INSTALL_COMFYUI_MANAGER'] = INSTALL_COMFYUI_MANAGER
OPTIONS['INSTALL_FLUX'] = INSTALL_FLUX
OPTIONS['INSTALL_FLUX_API'] = INSTALL_FLUX_API
OPTIONS['INSTALL_CUSTOM_NODES_DEPENDENCIES'] = INSTALL_CUSTOM_NODES_DEPENDENCIES

current_dir = !pwd
WORKSPACE = f"{current_dir[0]}/ComfyUI"

!pip install --upgrade gguf

%cd /mnt/workspace/

![ ! -d $WORKSPACE ] && echo -= Initial setup ComfyUI =- && git clone https://github.com/comfyanonymous/ComfyUI
%cd $WORKSPACE

if OPTIONS['UPDATE_COMFY_UI']:
  !echo "-= Updating ComfyUI =-"
  !git pull

# 安装FLUX-GGUF node
if OPTIONS['INSTALL_FLUX']:
  %cd custom_nodes
  !echo -= Initial setup FLUX =- && git clone https://github.com/city96/ComfyUI-GGUF

if OPTIONS['INSTALL_COMFYUI_MANAGER']:
  ![ ! -d ComfyUI-Manager ] && echo -= Initial setup ComfyUI-Manager =- && git clone https://github.com/ltdrdata/ComfyUI-Manager
  %cd ComfyUI-Manager
  !git pull

# 安装FLUX-API node
if OPTIONS['INSTALL_FLUX_API']:
  %cd ../
  ![ ! -d comfy-flux ] && echo -= Initial setup dashscope_api =- && git clone https://github.com/modelscope/comfyscope.git
  %cd comfy-flux
  !git pull

%cd $WORKSPACE

if OPTIONS['INSTALL_CUSTOM_NODES_DEPENDENCIES']:
  !pwd
  !echo "-= Install custom nodes dependencies =-"
  ![ -f "custom_nodes/ComfyUI-Manager/scripts/colab-dependencies.py" ] && python "custom_nodes/ComfyUI-Manager/scripts/colab-dependencies.py"

Model Download

#@markdown ###Download standard resources
%cd /mnt/workspace/ComfyUI
### FLUX1-DEV
!modelscope download --model=AI-ModelScope/FLUX.1-dev-gguf --local_dir ./models/unet/ flux1-dev-Q5_1.gguf

### clip
!modelscope download --model=AI-ModelScope/flux_text_encoders --local_dir ./models/clip/ t5xxl_fp16.safetensors
!modelscope download --model=AI-ModelScope/flux_text_encoders --local_dir ./models/clip/ clip_l.safetensors
!modelscope download --model=AI-ModelScope/flux_text_encoders --local_dir ./models/clip/ t5xxl_fp8_e4m3fn.safetensors


### vae
!modelscope download --model=AI-ModelScope/FLUX.1-dev --local_dir ./models/vae/ ae.safetensors

Running ComfyUI with cloudflared

!wget "https://modelscope.oss-cn-beijing.aliyuncs.com/resource/cloudflared-linux-amd64.deb"
!dpkg -i cloudflared-linux-amd64.deb

%cd /mnt/workspace/ComfyUI
import subprocess
import threading
import time
import socket
import urllib.request

def iframe_thread(port):
  while True:
      time.sleep(0.5)
      sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
      result = sock.connect_ex(('127.0.0.1', port))
      if result == 0:
        break
      sock.close()
  print("\nComfyUI finished loading, trying to launch cloudflared (if it gets stuck here cloudflared is having issues)\n")

  p = subprocess.Popen(["cloudflared", "tunnel", "--url", "http://127.0.0.1:{}".format(port)], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  for line in p.stderr:
    l = line.decode()
    if "trycloudflare.com " in l:
      print("This is the URL to access ComfyUI:", l[l.find("http"):], end='')
    #print(l, end='')


threading.Thread(target=iframe_thread, daemon=True, args=(8188,)).start()

!python main.py --dont-print-server

Load flowchart

flux1-dev flowchart:

https://modelscope.oss-cn-beijing.aliyuncs.com/resource/example_gguf_workflow.json

flux1 dashscope API flowchart:

https://github.com/modelscope/comfyscope/blob/main/workflow_examples/dashscope_flux.json

Click the link to directly access the Innovation Space experience

https://www.modelscope.cn/studios/AI-ModelScope/flux